Computer vision systems and methods for generating building models using three-dimensional sensing and augmented reality techniques

ABSTRACT

Computer vision systems and methods for generating building models using three-dimensional sensing and augmented reality (AR) techniques are provided. Image frames including images of a structure to be modeled are captured by a camera of a mobile device such as a smart phone, as well as three-dimensional data corresponding to the image frames. An object of interest, such as a structural feature of the building, is detected using both the image frames and the three-dimensional data. An AR icon is determined based upon the type of object detected, and is displayed on the mobile device superimposed on the image frames. The user can manipulate the AR icon to better fit or match the object of interest in the image frames, and can capture the object of interest using a capture icon displayed on the display of the mobile device.

RELATED APPLICATIONS

The present application claims the priority of U.S. ProvisionalApplication Ser. No. 63/165,972 filed on Mar. 25, 2021, the entiredisclosure of which is expressly incorporated herein by reference.

BACKGROUND Field of the Invention

The present disclosure relates to computer vision technology. Morespecifically, the present disclosure relates to computer vision systemsand methods for generating building models using three-dimensionalsensing and augmented reality techniques.

Related Art

In the insurance underwriting, building construction, solar, and realestate industries, computer-based systems for generating models ofphysical structures such as residential homes, commercial buildings,etc., is becoming increasingly important. In particular, in order togenerate an accurate model of a physical structure, one must have anaccurate set of data which adequately describes that structure.Moreover, it is becoming increasingly important to providecomputer-based systems which have adequate capabilities to generatefloor plans and layouts of both the interior and exterior features ofbuildings, as well as to identify specific features of such buildings(e.g., doors, windows, closets, etc.).

With the advent of mobile data capturing devices including phones,tablets and ground based vehicles, it is now possible to generate modelsof structures using such devices. Moreover, the types and quality ofsensors available in mobile devices continues to improve as timeprogresses. For example, today's mobile devices (e.g., smart phones) nowcome equipped with a wide array of sensors, such as cameras, gyroscopes,accelerometers, infrared sensors, and three-dimensional sensors such aslight detection and ranging (LiDAR) sensors. Even further, due to theincreased computing power of today's mobile devices, additionaltechnologies such as augmented reality (AR) are being integrated intosuch devices.

What would be desirable are computer vision systems and methods whichallow for the rapid generation of building models of structures,including interior and exterior models of such structures, which harnessthe very powerful three-dimensional sensing and AR capabilities oftoday's mobile devices. Accordingly, the computer vision systems andmethods of the present disclosure solve these, and other, needs.

SUMMARY

The present disclosure relates to computer vision systems and methodsfor generating building models using three-dimensional sensing andaugmented reality (AR) techniques. Image frames including images of astructure to be modeled (e.g., the interior or exterior of a building)are captured by a camera of a mobile device such as a smart phone, aswell as three-dimensional data corresponding to the image frames. Thethree-dimensional data could be captured using a three-dimensionalsensor of the mobile device, or computed from the image frames. Anobject of interest, such as a structural feature of the building (e.g.,wall, ceiling, floor, door, window, exterior surface, roof surface,etc.), is detected using both the image frames and the three-dimensionaldata. An AR icon is determined based upon the type of object detected,and is displayed on the mobile device superimposed on the image frames.The user can manipulate the AR icon to better fit or match the object ofinterest in the image frames. If the AR icon matches the object ofinterest (to the user's satisfaction), the user can capture the objectof interest using a capture icon displayed on the display of the mobiledevice. Successive objects of interest can be captured in the mannerdescribed above until all objects of interest have captured, whereupon acomplete model of the structure (e.g., a model of the interior orexterior of the structure) can be created.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing features of the invention will be apparent from thefollowing Detailed Description, taken in connection with theaccompanying drawings, in which:

FIG. 1 is a diagram illustrating hardware and software components of thesystems and methods of the present disclosure;

FIG. 2 is a flowchart illustrating processing steps carried out by thesystems and methods of the present disclosure;

FIGS. 3A-3B are screenshots illustrating wall capture functions andassociated wall capture user interface AR icons provided by the systemsand methods of the present disclosure;

FIG. 4 is a screenshot illustrating door capture functions andassociated door capture user interface AR icons provided by the systemsand methods of the present disclosure;

FIG. 5 is a screenshot illustrating window capture functions andassociated window capture user interface AR icons provided by thesystems and methods of the present disclosure;

FIG. 6 is a screenshot illustrating missing wall capture functions andassociated missing wall capture user interface AR icons provided by thesystems and methods of the present disclosure;

FIGS. 7A-7C are screenshots illustrating ceiling and floor capturefunctions and associated ceiling and wall capture user interface ARicons provided by the systems and methods of the present disclosure; and

FIGS. 8A-8I are screenshots illustrating exterior feature capturefunctions and associated exterior feature capture user interface ARicons provided by the systems and methods of the present disclosure.

DETAILED DESCRIPTION

The present disclosure relates to computer vision systems and methodsfor generating building models using three-dimensional sensing andaugmented reality (AR) techniques, as described in detail below inconnection with FIGS. 1-8I.

FIG. 1 is a diagram illustrating hardware and software components of thesystems and methods of the present disclosure, indicated generally at10. The system 10 includes a mobile device 12, which could be a smartcellular phone, a tablet computer, a laptop computer, a portablecomputer, or any other mobile computer system capable of beingtransported and used to inside of, or exterior to, a building. Themobile device 12 includes a processor 20 and a non-transitory,computer-readable memory 30 (e.g., one or more of volatile memory (e.g.,random access memory (RAM)) and/or non-volatile memory (e.g., flashmemory, electrically-erasable, programmable, read-only memory (EEPROM)),a memory chip, a memory card, etc.) on which is stored augmented realitybuilding and/or object modeling software code 32. The code 32 includescomputer-readable instructions which, when executed by the processor 20,cause the processor to perform the steps disclosed herein for modeling astructure and/or one or more objects in the structure usingthree-dimensional sensing and AR techniques.

The mobile device 12 also includes a camera 40 for capturing live video(in particular, a plurality of image frames which, together, constitutea video) and a three-dimensional sensor 50 for sensing three-dimensionaldata corresponding to an interior and/or exterior feature of a buildingor an object to be modeled. The sensor 50 could be, for example, a lightdetection and ranging (LiDAR) sensor or other type of three-dimensionalsensor which is capable of generating three-dimensional data, such as apoint cloud, corresponding to an interior and/or exterior feature of astructure or object to be imaged.

FIG. 2 is a flowchart illustrating processing steps carried out by thesystems and methods of the present disclosure, indicated generally at60. The steps 60 are embodied as the code 32 stored on the memory 30 ofFIG. 1 , and are executed by the processor 20 of the mobile device 12.Beginning in step 62, the device 12 acquires an image frame from thecamera 40. The image frame could be a single image frame (e.g., thecamera only takes a single image frame), or it could be one of a seriesof image frames being taken by the camera (e.g., an image frame from areal-time video taken by the camera 40). As will be discussed in greaterdetail in connection with FIGS. 3A-8I, the device 12 is pointed at aninterior or exterior feature of a structure or object to be modeled, andsuch that the camera 40 is pointed to the feature and the featureappears in the image frame captured by the camera 40. In step 64, thecamera acquires three-dimensional data corresponding to the featureusing the three-dimensional sensor 30 of the device 12. For example, ifthe three-dimensional sensor 30 is a LiDAR sensor, the three-dimensionaldata could be a three-dimensional point cloud that includesthree-dimensional data (points in a 3D coordinate system) correspondingto the feature displayed in the image frame captured by the camera 40.Thus, the feature is captured in steps 62 and 64 using bothtwo-dimensional data (e.g., a 2D image frame) and three-dimensional data(e.g., a 3D point cloud).

In step 66, the processor 20 processes the image frame and thethree-dimensional data to detect an object in the image frame, using theimage frame and the three-dimensional data. As will be discussed ingreater detail below in connection with FIGS. 3A-8I, the detected objectcould include, but is not limited to, an interior wall, ceiling, floor,door, or window, as well as an exterior feature such as a wall, door,window, or roof feature. Of course, the system 10 could be utilized todetect other objects, such as cabinets, furniture, fixtures, itemswithin a building, and/or exterior features such as patios, driveways,sidewalks, landscaping features, etc. Step 66 could be performed usingany suitable computer vision algorithm capable to detecting featuresfrom 2D and 3D data, such as a convolutional neural network (CNN) orother type of machine learning and/or artificial intelligence algorithmcapable to being trained to detect desired features in 2D and 3D data.

In step 68, the processor 20 determines the type of object detected.This could be accomplished using the algorithm discussed in step 66above, with one or more classification functions being performed. Forexample, in the case of a CNN, the CNN could output a binary classifierindicating whether the object detected is, or is not, a door. Multiplesuch classifications could be performed such that the algorithm canclassify various detected objects such as walls, ceilings, floors,doors, windows, exterior features, roof features, landscaping features,etc. In step 70, the processor 20 determines an augmented reality (AR)icon based on the type of object determined in step 68. For example, ifstep 68 determines that the detected object is a wall, in step 70, theprocessor 20 determines a suitable AR icon (e.g., a displayable userinterface (UI) element that can be manipulated by the user, such as a UIwidget or tool) corresponding to the wall. Of course, other AR icons canbe identified in step 70 corresponding to other types of objectsdetected in step 68 (e.g., a door AR icon if a door is detected, aceiling AR icon if a ceiling is detected, a floor AR icon if a floor isdetected, a window AR icon if a window is detected, etc.).

In step 72, the processor 20 causes a display of the device 12 todisplay the image frame as well as the AR icon superimposed on the imageframe. For example, if the image frame includes a wall, the image frame(including the wall) is displayed on the display of the device 12, witha wall AR icon superimposed on the wall in the image frame. The AR iconcould be superimposed on a portion of the detected object displayed inthe image frame, and/or it could be superimposed such that it appearsover the entire object displayed in the image frame.

In step 74, the processor 20 detects user input (e.g., the user takingan action in a user interface displayed on the device 12, such astapping and/or manipulating an icon) and determines whether user inputhas occurred. If a negative determination is made, control returns backto step 62 so that the foregoing processing steps can be repeated (e.g.,in real time, such that a real-time image frame is displayed along witha suitable AR icon which could move in the display so that it tracksmovement of the object in the image when the mobile device 12 is moved).If a positive determination is made, step 76 occurs, wherein theprocessor 20 determines whether the user input indicates that the userdesires to capture the object. In such circumstances, the user hadjudged that the AR icon that is displayed superimposed on the objectsufficiently represents the object, and that a model representation ofthe object should be created and added to a three-dimensional model. Ifso, step 78 occurs, wherein a model representation of the object iscreated by the processor 20 (e.g., a 3D plane corresponding to adetected wall is created) and added to a three-dimensional model createdby the processor 20 and stored in the memory 30. Otherwise, step 76occurs.

In step 80, a determination is made as to whether the user inputindicates that the user wishes to adjust the displayed AR icon. Forexample, the user may wish to adjust the displayed AR icon so that itbetter corresponds to the object over which it is displayed on thescreen, in order to ensure accurate modeling of the object. If apositive determination occurs, step 82 occurs, wherein the AR icon isadjusted based on user input (e.g., by the user repositioning the icon,changing its shape, etc.) and the adjusted AR icon is re-displayedsuperimposed over the object. Otherwise, step 84 occurs, wherein adetermination is made as to whether the current model of the feature ofinterest (e.g., a model of a room interior, a building exterior, etc.)is complete. If a negative determination is made, control returns tostep 62 so that additional objects can be detected and modeled in themanner described above. Otherwise, if a positive determination is made,step 86 occurs, wherein the processor 20 generates a complete model ofthe feature of interest, and processing ends. The complete model couldbe saved in the memory 30 of the device 12, and/or transmitted toanother computer system and/or program for further processing thereby(e.g., to generate an estimate of the cost of materials corresponding tothe modeled feature of interest, generate a report detailing the modeledfeature of interest, etc.).

FIGS. 3A-3B are screenshots illustrating wall capture functions andassociated wall capture user interface AR icons provided by the systemsand methods of the present disclosure. As can be seen in FIG. 3A, whenthe device 12 is pointed to a wall of interest (to be modeled), a userinterface screen 90 is displayed to the user, which includes an imageframe 92 that includes a wall of interest (captured using the camera 40of the device 12) and a wall AR icon 94 that can be manually orautomatically positioned superimposed on the wall of interest. A promptbar 96 requests that the user place the wall AR icon 94 on the wall ofinterest (in the event that the AR icon 94 is not automaticallypositioned on the wall of interest, or if the AR icon 94 requiresrepositioning in order to more accurately align with the wall ofinterest), and to actuate (e.g., tap) the capture button 98 when the ARicon 94 is accurately positioned on the wall of interest. Additionally,as can be seen in FIG. 3A, the user can switch to different AR icons fordifferent types of objects (as will be described below) by sliding aslider bar positioned above the capture button 98 (e.g., the user canswitch between a wall AR icon, a door AR icon, a missing wall AR icon,or engage a photo mode wherein photographs can be taken, if desired).

Alternatively and additionally, as illustrated in FIG. 3B, a differentwall capture AR tool could be provided for capturing walls. As shown inthe user interface screen 110, a line capture AR tool 112 is displayedsuperimposed on an image frame 112 and can positioned at a location inthe image frame 112 where no wall currently exists, but the user wishesto create a model of a dividing wall at that location. Examples of wheresuch functionality may be useful include “great rooms” that includekitchen, dining, and family areas that are open to each other, but theuser desires to separately capture and model such areas. When the device12 is pointed to a floor or a ceiling, the AR tool 112 is displayed as adot. Using the AR tool 112, the user can create a line where roomsshould be separated by capturing two dots anywhere along that line(whereupon the AR tool 112 illustrates the line as shown in FIG. 3B). Aprompt bar 116 provides instructions to the user as to how to utilizethe AR tool 112. When the wall has been captured, the user can actuatethe capture button 120, which causes the wall to be added to the modelbeing created. Additionally, when capturing of a room is complete, theuser can tap the “end room” button 122, which causes a model of the roomto be created based on the captured information delineated by the ARtool 112.

FIG. 4 is a screenshot illustrating door capture functions andassociated door capture user interface AR icons provided by the systemsand methods of the present disclosure. As can be seen, the userinterface screen 130 displays an image frame 132 that includes one ormore doors to be modeled, as well as one or more door AR icons 134A-134Bsuperimposed over the one or more doors. The AR icons 134A-134B allowthe user to identify one or more doors to be modeled, and can bemanipulated to define the doors. For example, each of the AR icons134A-134B could initially be displayed in the center of the screen 130as a dot which could be aligned with one corner of a door, whereupon theuser selects the corner as a starting corner by pressing a selectionbutton. The starting corner can be one of the corners on the top orbottom of the door, if desired. The device 12 could then be moved suchthat the dot is aligned with a corner opposite (diagonally) the startingcorner. A rectangle is then displayed on the screen (superimposed on thedoor), indicating the area that will be captured as a door. The capturebutton 138 can be actuated when the door has been marked, and the doorcould then be captured for subsequent inclusion in a model. As with theuser interface screens discussed herein, a prompt bar 136 could provideinstructions to the user as to how to capture the door, and when thedoor has been captured, the user can actuate the “End Room” button 140in order to end modeling of a room (e.g., when the door has beenadequately captured). As can be seen in FIG. 4 , various indiciaindicating measurements of the door (or other feature) could bedisplayed next to the AR icon, if desired. Additionally, the AR icons134A-134B could automatically “snap” (change shapes/sizes) to match adoor height and/or width (e.g., to automatically delineate common doorsizes). Further, the system could automatically identify the sizes andpositions of doors, if desired.

FIG. 5 is a screenshot illustrating window capture functions andassociated window capture user interface AR icons provided by thesystems and methods of the present disclosure. Specifically, as shown inFIG. 5 , the user interface screen 150 displays in image frame 152 thatincludes a window to be captured, and a window AR icon 154 displayedsuperimposed on the window. The window AR icon 154 can be manipulated bythe user to match the shape of the window. For example, the user canmark two opposite corners of the window, which cause the window AR icon154 to match the shape of the window to be captured. Further, the ARicon 154 could automatically identify the size and position of thewindow, if desired. When the AR icon 154 matches the shape of thewindow, the user can actuate the capture button 158 in order to capturethe window and add it to a model. As with the other screens describedherein, a prompt bar 156 is provided for instructing the user on how tomanipulate the AR icon 154, and an “End Room” button 160 is provided andcan be actuated when capturing of features of a room are complete, sothat the features can be added to the model.

FIG. 6 is a screenshot illustrating missing wall capture functions andassociated missing wall capture user interface AR icons provided by thesystems and methods of the present disclosure. As can be seen, the userinterface screen 170 displays an image frame 172 and a missing wall ARtool 174 superimposed on the image frame 172. The missing wall AR tool174 allows a user to define an opening in a wall for a model of astructure that is not a door or a window. This can be accomplished byusing the AR tool 174 to define two opposite corner points (in the samemanner as the door and window AR tools discussed above), whereupon theAR tool 174 defines a rectangular opening, or missing wall, that can beadded to a model. A prompt bar 176 guides the user as to how to utilizethe AR tool 174. When the missing wall has been defined using the ARtool 174, the user can actuate the capture button 178, whereupon themissing wall is added to a model. As with the other screens describedherein, a prompt bar 176 is provided for instructing the user on how tomanipulate the AR tool 174, and an “End Room” button 180 is provided andcan be actuated when capturing of features of a room are complete, sothat the features can be added to the model.

FIGS. 7A-7C are screenshots illustrating ceiling and floor capturefunctions and associated ceiling and wall capture user interface ARicons provided by the systems and methods of the present disclosure. Asshown in FIG. 7A, the user interface screen 190 displays an image frame192 that includes an image of a ceiling to be modeled, and a ceiling ARtool 194 superimposed on the image frame 192. The AR tool 194 can bere-positioned by the user as desired, to more accurately align with aceiling to be modeled. A prompt bar 196 provides instructions as to howto utilize the AR tool 194. When the AR tool 194 correctly aligns withthe ceiling shown in the image frame 192, the user can actuate thecapture button 198, whereupon the ceiling is added to a model. The “EndRoom” button 200 can be actuated once all desired features have beencaptured, to generate a model of a room.

Alternatively and additionally, ceilings can be modeled using adifferent AR tool shown in FIGS. 7B-7C. As shown in FIG. 7B, the userinterface screen 190 displays a floor AR tool 202 superimposed on thefloor of a room shown in an image frame, when the device 12 is pointedat the floor of a room. The AR tool 202 can be aligned as desired, andonce aligned, the user can actuate a selection button to mark a point onthe floor. Then, as shown in FIG. 7C, the user can point the device 12to the ceiling of the same room, whereupon a ceiling AR tool 204 appearsin the display superimposed on the ceiling shown in the image frame.Additionally, a vertical AR tool 206 is displayed, interconnecting theceiling AR tool 204 with the floor AR tool 202. When the ceiling AR tool204 is properly positioned, the user can actuate a selection button,which causes the device 12 to calculate a ceiling height based on thepositions of the floor AR tool 202 and the ceiling AR tool 204.

FIGS. 8A-8I are screenshots illustrating exterior feature capturefunctions and associated exterior feature capture user interface ARicons provided by the systems and methods of the present disclosure. Itis noted that all of the AR tools and associated functions discussedabove with capturing of interior features could also be used to captureexterior building features. As shown in FIG. 4A, a user interface screen210 displays an image frame 212 including an image of one or moreexterior features to be modeled, and one or more AR icons 214 displayedsuperimposed on the image 212. The one or more AR icons 214 couldcorrespond to any of the AR icons discussed above, as well as additionalicons discussed hereinbelow. When desired exterior features have beencaptured, the user can actuate a capture button 218 which causes thefeatures to be added to a model. A prompt bar 216 provides the user withinstructions on how to operate the one or more AR icons 214, and theuser can actuate the “End Room” button when all desired exteriorfeatures have been captured, in order to generate a model of theexterior of the building. The tools described in FIGS. 8A-8I can be usedto capture exterior walls, doors, windows, building footprints, exteriorwall heights, and other features.

As shown in FIG. 8B, the AR icon 214 could allow a user to define pointsalong a wall base of the exterior of a building, and could function in amanner similar to the wall AR tool 114 discussed above in connectionwith FIG. 3B. The tool 114 could be aligned with the base of an exteriorwall and used to mark the base (typically, at the top of the foundationof the structure). Other features, such as the ground, could also bemarked using the tool 214. A default base elevation could be definedusing the tool 214, as well as a bottom horizontal line for capturingexterior walls (described below). Advantageously, the tool 214 allowsthe user to capture walls on a stepped foundation, as well as standardwalls.

As shown in FIG. 8C, the tool 214 could also allow for marking andcapturing of wall top lines, for indicating the tops of exterior walls.As the tool 214 is moved from point to point along the top of a wall, aline is indicated to the user. The user aligns the line with the topline of the wall and marks a second point. The ends points of the linedo not need to be positioned at the ends of the wall, and could bepositioned at any point along the wall top. The corresponding wall isthen adjusted to match the captured line. This allows the user tocapture gable end walls (triangular in shape) was well as standard walls(rectangular in shape).

As shown in FIG. 8D, the tool 214 could be utilized to mark the tops ofwalls near roof features. To do so, the tool 214 could be split into twoor more soffit AR tools 214A-214B, which could be positioned so thatthey are superimposed over the eaves of the roof. A prompt 216 couldprompt the user to mark the eaves by positioning the AR tools 214A-214Bon the eaves, and then actuating a selection button. As shown in FIG.8E, once the eaves have been marked, the tools 214A-214B could projectmarker lines to assist with lining the tools 214A-214B with the tops ofthe walls in the image frame. Once aligned, the user can actuate acapture button to mark the wall tops.

It is noted that information about walls captured by the system could beutilized to automatically construct roof features. Such a feature ishelpful in circumstances where the three-dimensional sensor 40 is notcapable of detecting three-dimensional data corresponding to rooffeatures (e.g., the sensor is not sensitive enough, or the roof featuresare not within range of the sensors). As shown in FIG. 8F, a selectionindicator 230 is displayed on the screen, and the user can be promptedto identify the type of wall that appears in the screen, such as a gablewall (triangle shape) or an eve wall (rectangle shape). Once the walltype is identified, the system can automatically create the roofstructure above the walls based on the identified wall types. Forexample, a roof plane could be constructed by the system at the top ofeach identified eave wall, and could be offset from the wall to allowfor overhangs. The slope could be calculated from adjacent gable wallsthat have been identified. If the adjacent walls are not gable walls, adefault slope could be utilized or the user could specify a slope value.

As shown in FIG. 8G, each of the roof planes (indicated at 232 and 234)could be intersected with one another to form closed polyhedrons 236 incombination with the walls. The “straight skeleton” methods described inU.S. Pat. No. 10,127,670 to Lewis, et al., the entire disclosure ofwhich is expressly incorporated herein by reference, could be utilizedto form the polyhedrons 236, as well as constructive solid geometry(CSG) methods.

As shown in FIG. 8H, the tool 214 could also allow a user to define oneor more roof features of a building by selecting roof faces and to makeslope adjustments as needed. The slope could be set manually (the usermanually measures and enters the slope), or by placing the tool 214 onthe desired roof surface and actuating capture button (similar tocapturing walls or ceilings as discussed above). As shown in FIG. 8I,the roof surface could be delineated by a marker 240 superimposed overthe roof surface.

Having thus described the system and method in detail, it is to beunderstood that the foregoing description is not intended to limit thespirit or scope thereof. It will be understood that the embodiments ofthe present disclosure described herein are merely exemplary and that aperson skilled in the art may make any variations and modificationwithout departing from the spirit and scope of the disclosure. All suchvariations and modifications, including those discussed above, areintended to be included within the scope of the disclosure. What isdesired to be protected by Letters Patent is set forth in the followingclaims.

What is claimed is:
 1. A computer vision system for generating a modelof a building, comprising: a mobile device including a three-dimensionalsensor, a camera, and a processor; and a memory in communication withthe processor and storing modeling code, the processor executing themodeling code to perform the steps of: acquiring an image frame from thecamera, the image frame depicting at least part of a building to bemodeled; acquiring three-dimensional data from the three-dimensionalsensor; processing the image frame and the three-dimensional data todetect a building object in the building frame; determining an augmentedreality (AR) icon based on a type of the building object; displaying theAR icon on a display of the mobile device superimposed on the imageframe, the AR icon adjustable relative to the building object displayedin the image frame; allowing a user to capture the building object usingthe AR icon; and generating a complete model of the building includingthe building object.
 2. The system of claim 1, wherein the processorfurther performs the steps of allowing the user to adjust the AR iconand re-displaying the AR icon after adjustment.
 3. The system of claim1, wherein the building object is a wall and the AR icon is a wall ARicon displayed superimposed on the wall.
 4. The system of claim 1,wherein the building object is a door and the AR icon is a door AR icondisplayed superimposed on the door.
 5. The system of claim 1, whereinthe building object is a window and the AR icon is a window AR icondisplayed superimposed on the window.
 6. The system of claim 1, whereinthe AR icon is a missing wall AR icon.
 7. The system of claim 6, whereinthe missing wall AR icon is positionable in a desired location in theimage frame to define a wall.
 8. The system of claim 1, wherein thebuilding object is a ceiling and the AR icon is a ceiling AR icondisplayed superimposed on the ceiling.
 9. The system of claim 8, whereinthe ceiling AR icon is adjustable to measure a ceiling height.
 10. Thesystem of claim 1, wherein the building object is a floor and the ARicon is a floor AR icon displayed superimposed on the floor.
 11. Thesystem of claim 1, wherein the building object is an eave and the ARicon is an eave AR icon displayed superimposed on the eave.
 12. Thesystem of claim 1, wherein the building object is a roof face and the ARicon is a roof face icon displayed superimposed on the roof face.
 13. Acomputer vision method for generating a model of a building, comprisingthe steps of: acquiring an image frame from a camera of a mobile device,the image frame depicting at least part of a building to be modeled;acquiring three-dimensional data from a three-dimensional sensor of themobile device; processing the image frame and the three-dimensional datato detect a building object in the building frame; determining anaugmented reality (AR) icon based on a type of the building object;displaying the AR icon on a display of the mobile device superimposed onthe image frame, the AR icon adjustable relative to the building objectdisplayed in the image frame; allowing a user to capture the buildingobject using the AR icon; and generating a complete model of thebuilding including the building object.
 14. The method of claim 13,wherein the processor further performs the steps of allowing the user toadjust the AR icon and re-displaying the AR icon after adjustment. 15.The method of claim 13, wherein the building object is a wall and the ARicon is a wall AR icon displayed superimposed on the wall.
 16. Themethod of claim 13, wherein the building object is a door and the ARicon is a door AR icon displayed superimposed on the door.
 17. Themethod of claim 13, wherein the building object is a window and the ARicon is a window AR icon displayed superimposed on the window.
 18. Themethod of claim 13, wherein the AR icon is a missing wall AR icon. 19.The method of claim 18, wherein the missing wall AR icon is positionablein a desired location in the image frame to define a wall.
 20. Themethod of claim 13, wherein the building object is a ceiling and the ARicon is a ceiling AR icon displayed superimposed on the ceiling.
 21. Themethod of claim 20, wherein the ceiling AR icon is adjustable to measurea ceiling height.
 22. The method of claim 13, wherein the buildingobject is a floor and the AR icon is a floor AR icon displayedsuperimposed on the floor.
 23. The method of claim 13, wherein thebuilding object is an eave and the AR icon is an eave AR icon displayedsuperimposed on the eave.
 24. The method of claim 13, wherein thebuilding object is a roof face and the AR icon is a roof face icondisplayed superimposed on the roof face.